Method and system for three dimensional digital holographic aperture synthesis

ABSTRACT

Laser 3D imaging techniques include splitting a laser temporally-modulated waveform of bandwidth B and duration D from a laser source into a reference beam and a target beam and directing the target beam onto a target. First data is collected, which indicates amplitude and phase of light relative to the reference beam received at each of a plurality of different times during a duration D at each optical detector of an array of one or more optical detectors perpendicular to the target beam. Steps are repeated for multiple sampling conditions, and the first data for the multiple sampling conditions are synthesized to form one or more synthesized sets. A 3D Fourier transform of each synthesized set forms a digital model of the target for each synthesized set with a down-range resolution based on the bandwidth B.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims benefit of Provisional Appln. 62/183,921, filedJun. 24, 2015, the entire contents of which are hereby incorporated byreference as if fully set forth herein, under 35 U.S.C. §119(e).

STATEMENT OF GOVERNMENTAL INTEREST

This invention was made with government support under Contract Nos.FA8650-14-M-1793, FA8650-14-M-1787, and FA8650-15-C-1871 awarded by theDepartment of the Air Force. The government has certain rights in theinvention.

BACKGROUND

Light Detection And Ranging, abbreviated LADAR herein, is a surveyingtechnology that measures distance by illuminating a target with a laserlight and timing the return, and the term has been coined as an analogyfor the previously well-known technology of radio detection and ranging(RADAR). Three dimensional (3D) imaging based on LADAR has been arapidly expanding research field with approaches that capitalize onparallel readout architectures seeing significant advancement recently.For example, in flash LADAR systems, the target is broadly illuminatedwith a high energy pulsed laser and a receive aperture is used to imagethe target onto a fast focal-plane array (FPA) of optical detectors. Thefast FPA typically consists of an array of avalanche photodiodes (APDs)coupled to an advanced read-out-integrated-circuit (ROIC), which alsoallows it to time resolve the return pulses. Several companies nowmanufacture fast FPAs operating either in linear or Geiger mode forshort and long range flash LADAR respectively. Over the past decade,incoherent, direct-detect flash LADAR has been used for both terrestrialand airborne applications including mapping and autonomous navigation.

Synthetic aperture LADAR (SAL), distributed or sparse aperture imagingand holographic aperture LADAR (HAL) are coherent imaging techniquesthat coherently combine spatially and temporally diverse target returnsto overcome the conventional diffraction limit. By recording individualestimates of the electric field with either a single optical detector oran optical detector array, and by considering the relative motionbetween the transceiver and the target, these field estimates can besynthesized or “stitched” in the pupil plane to produce enhancedresolution two dimensional (2D) imagery. The pupil plane is the plane inwhich light first impinges on the optics of a system, before anyfocusing of the impinging rays. To form 3D imagery, two or morebaselines in an interferometric SAL system, or two or more wavelengthsin a distributed aperture or HAL system have been proposed anddemonstrated. The unresolved dimension is derived from the measuredphase difference between the spatial or wavelength multiplexed 2Dimages. Because these techniques rely on a few synthetic wavelengths,targets with significant structure or discontinuities are poorlyrendered. Recent work extended the number of discrete wavelengths to 256to address these issues.

SUMMARY

The achievable cross-range resolution in flash LADAR is limited by theconventional diffraction limit imparted by the receive aperture size,while the down-range resolution is limited by the effective bandwidth ofthe LADAR system. These resolution constraints typically limit theapplicability of flash LADAR especially when ultra-high resolution isrequired, such as in large-volume metrology applications. Targets withsignificant structure or discontinuities are still not satisfactoryrendered by the SAL and HAL techniques for some applications, such as inlarge-volume metrology applications, even with the larger number ofwavelengths attempted to date.

Techniques are provided to demonstrate for the first time, fullydown-range resolved coherent imaging with aperture synthesis. In someembodiments, this technique is called three dimensional holographicaperture LADAR (3D-HAL) for brevity; but, does not require the twodimensional array of detectors required in HAL. For example, thetransverse sampling can be achieved with a single point detector ifprecise (with precision small compared to the transverse extent of thedetector) measurement of the transverse location of the detector isavailable at all sample points. With an array, it is possible to samplefield segments with some overlap which can then be used for registrationof the field. Herein the technique is called, more generally, highresolution 3D LADAR.

In a first set of embodiments, a method includes step a for splitting alaser temporally-modulated waveform of bandwidth B and duration D from alaser source into a reference beam and a target beam and step b fordirecting the target beam onto a target. The method includes step c forcollecting first data that indicates amplitude and phase of lightrelative to the reference beam received at each of a plurality ofdifferent times during a duration D at each optical detector of an arrayof one or more optical detectors in a plane perpendicular to the targetbeam. The method also includes step d for repeating steps a, b and c formultiple sampling conditions, and step e for synthesizing the first datafor the multiple sampling conditions to form one or more synthesizedsets. The method still further includes step f for performing a 3DFourier transform of each synthesized set to form a digital model of thetarget for each synthesized set with a down-range resolution based onthe bandwidth B. In some embodiments, the method also includes step gfor operating a display device based at least in part on at least aportion of the digital model of the target for at least one synthesizedset.

In some embodiments of the first set of embodiments, synthesizing thefirst data further comprises, for each synthesized set, selecting aplurality of subsets of the first data, synthesizing each subsetseparately to produce a synthesized subset and incoherently combiningthe plurality of synthesized subsets.

In some embodiments of the first set of embodiments, performing a 3DFourier transform of each synthesized set further comprises performing aone dimensional Fourier Transform of each dimension independently andcombining results from all dimensions. In some of these embodiments,each dimension is transformed in separate sub-sections to further limitmemory overhead consumption.

In some embodiments of the first set, the array of one or more opticaldetectors is a subset of pixels in a digital camera to allow a framerate for the subset of pixels to be greater than a frame rate for allthe pixels in the digital camera. In these embodiments, repeating stepsa, b, and c for the plurality of sampling conditions includes repeatingsteps a, b, and c for a plurality of different subsets of the pixels inthe digital camera.

In some of the first set of embodiments, the method also includesdetermining an average range to the target based on a travel time of alaser pulse reflected from the target and providing a reference pathlength for the reference beam based on the average range to the target.

In some of the first set of embodiments, the digital model is a pointcloud and the display device is a system configured to render a surfacefrom a point cloud.

In various embodiments of the first set of embodiments, the displaydevice is either a system configured to identify an object based on thedigital model, or a system configured to operate on the target based onthe digital model, or both.

In some of the first set of embodiments, the pluralities of samplingconditions are a plurality of different angles from the target to thearray of one or more optical detector.

In some of the first set of embodiments, the plurality of samplingconditions is a plurality of different times while the target issubjected to a change in environment or while one target is replaced byanother.

In some of the first set of embodiments, the one or more synthesizedsets includes at least two synthesized sets; and, the step of operatingthe display device further comprises operating the display device topresent second data that indicates a difference between at least twodifferent digital models formed from the at least two synthesized sets.In some of these embodiments, the synthesized sets represent shapes ofan object for corresponding different sampling conditions (e.g.,different times or different angles, or different instances of anassembly line of similar targets). In some of these latter embodiments,the change in environment is a change in thermal conditions and thedifference between the at least two different digital models indicatesthermal expansion in response to the change in thermal conditions.

In other sets of embodiments, a computer-readable medium or an apparatusor a system is configured to perform one or more steps of one or more ofthe above methods.

Still other aspects, features, and advantages are readily apparent fromthe following detailed description, simply by illustrating a number ofparticular embodiments and implementations, including the best modecontemplated for carrying out the invention. Other embodiments are alsocapable of other and different features and advantages, and its severaldetails can be modified in various obvious respects, all withoutdeparting from the spirit and scope of the invention. Accordingly, thedrawings and description are to be regarded as illustrative in nature,and not as restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments are illustrated by way of example, and not by way oflimitation, in the figures of the accompanying drawings in which likereference numerals refer to similar elements and in which:

FIG. 1 is a block diagram that illustrates example operation of a highresolution 3D LADAR system, according to an embodiment;

FIG. 2 is a block diagram that illustrates example components of a highresolution 3D LADAR system, according to an embodiment;

FIG. 3 is a block diagram that illustrates example processing of datafrom operation of a high resolution 3D LADAR system, according to anembodiment;

FIG. 4 is a flow diagram that illustrates an example method of using ahigh resolution 3D LADAR system, according to an embodiment;

FIG. 5A is a block diagram that illustrates an example 2D array ofoptical detectors forming a 2D synthetic aperture, according to anembodiment;

FIG. 5B is a plot that illustrates an example 3D image using a singleinstance of the 2D array of FIG. 5A, according to an embodiment;

FIG. 5C is a plot that illustrates an example 3D image using multipleinstances of the 2D array of FIG. 5A in a synthetic aperture, accordingto an embodiment;

FIG. 6A is a photograph that illustrates an example surface used as atarget, according to an embodiment;

FIG. 6B is a plot that illustrates an example 3D model of the surface ofFIG. 6A using a single instance of a 2D array, according to anembodiment;

FIG. 6C is a plot that illustrates an example 3D model of the surface ofFIG. 6A using multiple instances of the 2D array in a syntheticaperture, according to an embodiment;

FIG. 7A is a photograph that illustrates an example object used as atarget, according to an embodiment;

FIG. 7B is a plot that illustrates an example 3D model of the object ofFIG. 7A using a single instance of a 2D array, according to anembodiment;

FIG. 7C is a plot that illustrates an example 3D model of the object ofFIG. 7A using multiple instances of the 2D array in a syntheticaperture, according to an embodiment;

FIG. 8A is a photograph that illustrates another example object used asa target, according to an embodiment;

FIG. 8B is a plot that illustrates an example 3D model of the object ofFIG. 8A using a single instance of a 2D array, according to anembodiment;

FIG. 8C is a plot that illustrates an example 3D model of the object ofFIG. 8A using multiple instances of the 2D array in a syntheticaperture, according to an embodiment;

FIG. 9A is a photograph that illustrates another example object used asa target, according to an embodiment;

FIG. 9B is an image that illustrates an example distribution ofdifferences in the object of FIG. 9A at different times due to differentenvironmental conditions, according to an embodiment;

FIG. 9C is a plot that illustrates components of expansion toward andaway from an array of optical detectors, according to an embodiment;

FIG. 9D is an image that illustrates an example distribution ofdifferences in the object of FIG. 9A at different times due to differentenvironmental conditions corrected for assumed direction of expansion,according to an embodiment;

FIG. 10 is a block diagram that illustrates a computer system 1 uponwhich an embodiment of the invention may be implemented; and

FIG. 11 illustrates a chip set upon which an embodiment of the inventionmay be implemented.

DETAILED DESCRIPTION

A method and apparatus are described for high resolution 3D LADAR,including 3D-HAL. In the following description, for the purposes ofexplanation, numerous specific details are set forth in order to providea thorough understanding of the present invention. It will be apparent,however, to one skilled in the art that the present invention may bepracticed without these specific details. In other instances, well-knownstructures and devices are shown in block diagram form in order to avoidunnecessarily obscuring the present invention.

Notwithstanding that the numerical ranges and parameters setting forththe broad scope are approximations, the numerical values set forth inspecific non-limiting examples are reported as precisely as possible.Any numerical value, however, inherently contains certain errorsnecessarily resulting from the standard deviation found in theirrespective testing measurements at the time of this writing.Furthermore, unless otherwise clear from the context, a numerical valuepresented herein has an implied precision given by the least significantdigit. Thus a value 1.1 implies a value from 1.05 to 1.15. The term“about” is used to indicate a broader range centered on the given value,and unless otherwise clear from the context implies a broader rangaround the least significant digit, such as “about 1.1” implies a rangefrom 1.0 to 1.2. If the least significant digit is unclear, then theterm “about” implies a factor of two, e.g., “about X” implies a value inthe range from 0.5X to 2X, for example, about 100 implies a value in arange from 50 to 200. Moreover, all ranges disclosed herein are to beunderstood to encompass any and all sub-ranges subsumed therein. Forexample, a range of “less than 10” can include any and all sub-rangesbetween (and including) the minimum value of zero and the maximum valueof 10, that is, any and all sub-ranges having a minimum value of equalto or greater than zero and a maximum value of equal to or less than 10,e.g., 1 to 4.

Some embodiments of the invention are described below in the context of3D-HAL at 2 kilometer ranges with centimeter down-range resolution, andat 250 centimeter ranges with micron down-range resolution formetrology. However, the invention is not limited to this context. Inother embodiments, stationary arrays are used with high resolution 3DLADAR processing to monitor objects in time, e.g., to compare similarobjects coming off an assembly line; to monitor stresses in objectsimposed by harsh environmental conditions, such as at high and lowtemperatures or at high and low pressures and in the presence ofcorrosive chemical environments, or some combination; to use stationaryor moving arrays to form digital representations used by other system toclassify or identify objects, or control robots to operate on or avoidobjects, or weapons systems to attack or avoid objects, or surveillancesystem to rescue people, animals or other objects, or for space basedimaging, among other applications.

1. OVERVIEW

In 3D-HAL, a temporally-modulated waveform (a form of frequency or phasemodulated continuous wave, FMCW or PMCW) is introduced for ranging ateach aperture location, which is not used in HAL. In many illustratedembodiments, the temporal modulation is a linear frequency change and iscalled a chirp. In addition heterodyne detection is used with the samelaser temporally-modulated waveform as a reference. While any coherentform of laser ranging could be utilized, FMCW chirped heterodyne ranginghas the advantage of providing very high downrange resolution with areduced receiver bandwidth via optical stretched processing. Theachievable range resolution, δ_(R), of this technique is limited by thechirp bandwidth, B, and is described by Equation 1.

δ_(R) =c/(2B)  (1)

where c is the speed of light.

FIG. 1 is a block diagram that illustrates example operation of a highresolution 3D LADAR system, according to an embodiment. A chirpedtransmit source 110 produces transmitted light (Tx) 112 that floodilluminates a target 190 at some nominal range R, and the fast focalplane array (FPA) records the interference between the returned light(Rx) 192 and a reference beam chirp produced concurrently with thetransmitted light Tx. The complex return field g₀(x,y,t) 193 resultingfrom the heterodyne interference encodes the 3-D spatial information atvarious positions 130 a through 130 g, collectively referenced aspositions 130, along a line of travel 121. The total synthetic aperture,d_(SA) 123, provides enhanced resolution along the direction of platformmotion along line of travel 121. The chirp provides enhanced down-rangeresolution to allow 3D rendering of the target 190. The data collectedis referred to as a field segment, g₀ (x,y,t_(f)), where x and y arecross-range coordinates and t_(f) is the time during which the temporalinterference is recorded (i.e. “fast time” in SAL formalism), which isshort compared to the chirp duration D. Throughout this paper, sensorarrays that are capable of recording this temporal interference arecalled “fast FPAs” to distinguish them from more traditional sensorarrays that operate with very slow frame rates. However, in 3D-HAL thefast FPAs need not be restricted to being placed in the “focal plane” asthe acronym suggests. But can be placed in the pupil plane as shown inFIG. 1.

FIG. 2 is a block diagram that illustrates example components of a highresolution 3D LADAR system, according to an embodiment. A laser source212 emits a carrier wave 201 that is frequency modulated in frequencymodulator 214 to produce a chirp of time dependent frequency that has abandwidth B and a duration D. A splitter 218 splits the chirp into atarget beam 205 with most of the energy of the beam 203 and a referencebeam 207 with a much smaller amount of energy that is nonetheless enoughto produce good heterodyne interference with the returned light 291scattered from the target 190. The reference beam is delayed in areference path 220 sufficiently to arrive at the detector array 230 withthe scattered light. In various embodiments, from less to more flexibleapproaches, the reference is caused to arrive with the scattered orreflected field by: 1) putting a mirror in the scene to reflect aportion of the transmit beam back at the detector array so that pathlengths are well matched; 2) using a fiber delay to closely match thepath length and broadcast the reference beam with optics near thedetector array, as suggested in FIG. 2, with or without a path lengthadjustment to compensate for the phase difference observed or expectedfor a particular range; or, 3) using a frequency shifting device(acousto-optic modulator) or time delay of a local oscillator waveformmodulation to produce a separate modulation to compensate for pathlength mis-match; or some combination.

The detector array is a single detector or a 1D or 2D array of detectorsarranged in a plane perpendicular to target beam 205. The phase andamplitude of the interface pattern is recorded by acquisition system 240for each detector at multiple times during the pulse chirp duration D.The number of temporal samples per chip duration affects the down-rangeextent. The number is often a practical consideration chosen based onchirp repetition rate and available camera frame rate. The frame rate isthe sampling bandwidth, often called “digitizer frequency.”. Basically,if X number of detector array frames are collected during a chirp withresolution bins of Y range width, then a X*Y range extent can beobserved. The acquired data is made available to a processing system250, such as a computer system described below with reference to FIG.10, or a chip set described below with reference to FIG. 11. A 3D-HALmodule determines the digital model of the target based on the acquireddata. Any known apparatus or system may be used to implement the lasersource 212, frequency modulator 214, beam splitter 216, reference path220, detector array 230, or acquisition system 240. Optical coupling toflood or focus on the target or focus past the pupil plane are notdepicted.

For example, in some embodiments, the laser used was actively linearizedwith the modulation applied to the current driving the laser.Experiments were also performed with electro-optic modulators providingthe modulation. The system is configured to produce a chirp of bandwidthB and duration D, suitable for the down-range resolution desired, asdescribed in more detail below for various embodiments. For example, insome illustrated embodiments, a value of B about 90 GHz and D of about200 ms were chosen to work within the confines of the relatively lowdetector array frame rate in the experiments performed. These choiceswere made to observe a reasonably large range window of about 30 cm,which is often important in shape of an object and identification of theobject. Although processes, equipment, and data structures are depictedin FIG. 1 as integral blocks in a particular arrangement for purposes ofillustration, in other embodiments one or more processes or datastructures, or portions thereof, are arranged in a different manner, onthe same or different hosts, in one or more databases, or are omitted,or one or more different processes or data structures are included onthe same or different hosts.

Aperture synthesis depends on translation between the speckle field,that is, g0 the interference pattern of the backscattered light with thereference beam, and the receive aperture in the aperture plane, which inthis depicted configuration is also the pupil place. A multitude ofconfigurations achieve this effect. In the example analysis, thetransceiver locations 130 are located and it is assumed that thetransmitter moves with the fast FPA in a monostatic configuration (bothtransmitter and receivers moving together). However, this assumption isnot required as bi-static (one is stationary and the other movingrelative to the target) and multi-static (both moving relative to thetarget or several detectors or transmitters moving separately)configurations are also possible. The spatial relationship betweentransmitter and receiver does not have to be known to form imagery.However, knowledge of the spatial relationship can help in processingand interpreting the image. The synthetic aperture size, d_(SA) 123, isshown and the enhanced cross-range resolution is nominally given byEquation 2

δ_(CR) =λR/(2d _(SA) +d _(AP))  (2)

where λ is the carrier wavelength, R is the range to target, d_(AP) isthe focal plane array width in the dimension of travel, and the stepsize is assumed to be d_(AP)/2.

FIG. 3 is a block diagram that illustrates example processing of datafrom operation of a high resolution 3D LADAR system, according to anembodiment. FIG. 3 shows the corresponding sequence of chirp waveforms.In graph 310, the horizontal axis 312 indicates time increasing to theright in arbitrary units; and, the vertical axis 314 indicates opticalfrequency in the target beam, increasing upward in arbitrary units. Ateach aperture location 130 along the line of platform motion, a chirp isemitted with duration D 318 and bandwidth B 316. An example recordedfield segment for aperture location “b” is drawn as a small cube 320 b.It is important to note that the fast FPA samples the field spatially atthe pixel spacing in the aperture plane, while the frame rate of thefast FPA gives the temporal sampling rate that provides the down-ranginformation. The pixel spacing sets the overall field-of-view of thesystem in the cross-range dimensions, while the frame rate isproportional to the maximum range difference that can be measured in thedown-range dimension.

Each recorded field segment 320 a through 320 g is subsequentlysynthesized during a post-processing step with the other field segments,resulting in a larger synthesized or fully registered field estimate asshown graph 330. The horizontal axis 332 indicates distance x in thedirection of platform motion; the vertical axis 334 indicates distance yin the perpendicular cross-range direction; and, the third axis 335indicates samples in the time domain that indicates down-range distance.If the field segments overlap in the direction of platform motion, thenthe overlap provides advantages for registering the cubes to each otherspatially. This synthesized segments structure is call a “data cube”herein. Synthesizing can either be performed by measuring the aperturepositions with high precision or by using data driven registrationtechniques with sufficient overlap between field segments. Even afterregistration, the field segments typically remain incoherent with eachother because the aperture motion must technically be compensated toless than the optical carrier wavelength, which is often not physicallypossible. Phase gradient autofocus or prominent point algorithms providecoherence across field segments and the data can be further enhancedusing digital image correction for atmospheric or other aberrationsusing any method known at the time the process is implemented.

By transforming the whole data cube, with a 3D FFT, all dimensions arecompressed into a fully resolved 3D distribution. Once fully compressed,peak extraction is performed along the downrange dimension to constructa point cloud. The 3D FFT generates a 3D array where the magnitude ateach array element represents the amount of energy reflected from thatpoint in space. To extract the 3D data from this 3D array for pointcloud visualization purposes, the 1-D array existing at each transverselocations (other 2 dimensions) in this 3D array was analyzed. Theanalysis of the 1-D array located and fit any peaks present. These werelogged as 3D points in the point cloud.

FIG. 4 is a flow diagram that illustrates an example method of using ahigh resolution 3D LADAR system, according to an embodiment. Althoughsteps are depicted in FIG. 4 as integral steps in a particular order forpurposes of illustration, in other embodiments, one or more steps, orportions thereof, are performed in a different order, or overlapping intime, in series or in parallel, or are omitted, or one or moreadditional steps are added, or the method is changed in some combinationof ways.

In step 401, a transceiver is configured to transmit a frequencymodulated continuous wave (FMCW) laser beam with bandwidth B andduration D, called a laser chirp, and to receive in a spatial array ofone or more optical detectors arranged in a plane perpendicular to thetransmit beam at a temporal sampling rate, called a “fast frame rate,”that is short compared to the duration D of the chirp. The bandwidth Bis chosen to resolve the down-range features of the desired targetaccording to Equation 1 and the duration is chosen to be short comparedto the time for the platform to travel a distance on the order of thewidth of the array of detectors in the direction of travel, d_(AP), ifany. The transceiver is also configured to produce a reference laserchirp to interfere with the returned signal at the detector array, e.g.,using any of the methods described above. This provides highly precisephase information.

In step 411, it is determine whether some condition is satisfied to takethe next sample of the target, where a sample includes the full laserchirp and the fast frame rate temporal sampling of the return to providea full field segment. In some embodiments, the condition is that theplatform has moved a distance on the order of the d_(PA) (or about halfthat distance to provide for overlap that is useful in registering thesuccessive field segments). In some embodiments in which a stationaryarray is observing a rotating target, the condition is that the targethas rotated a circumferential distance of about d_(PA) (or about half ofthat). In some embodiments in which a stationary array is observing aseries of items on an assembly line, the condition is that the targetreached a position on the assembly line for imaging. In someembodiments, in which a fixed array is observing a target subjected tosome environmental stress, the condition is that sufficient time haspassed to achieve a noticeable effect from the stress or that someenvironmental parameter has reached a certain value. If the conditionfor another sample has not been satisfied, control passes to step 413 tomove the platform the desired distance or wait for the desired time orfor the desired environmental parameter value. In some embodiments,steps 411 and 413 is performed by a hardware controller for the lasersource; and, in some embodiments, steps 411 and 413 is performed by ageneral purpose processor programmed to control the laser source.Control then passes back to step 411.

In some embodiments, novel steps were developed to accumulate usefulsamples during step 413. For example, in some embodiments when there isno physical motion of the aperture, the receiver array “motion” isachieved by shifting a sub-region (e.g., 200×280 pixels out of a1000×1960pixel≈2 Megapixel camera) of the camera array which wasactively being sampled. This offers an advantage because only collectingdata from a sub-set of a camera array can allow faster frame rates. Insome embodiments, the speckle averaging is achieved by rotating thetarget (which causes a translation of the speckle field incident on thedetector array) by an amount desired to get a new speckle realization.An entirely new data set was then collected. The speckle averaging wasperformed by incoherently summing (amplitude not phase) the 3D resolveddata cube prior to point cloud extraction. The use of multiple imagingmodes (aperture translation, target rotation) to achieve independentspeckle observations for incoherent averaging offers the advantage ofreduced noise in the point cloud. Thus, in some embodiments, this stepincludes synthesizing the first data by, for each synthesized set,selecting a plurality of subsets of the first data, synthesizing eachsubset separately to produce a synthesized subset and incoherentlycombining the plurality of synthesized subsets

If it is determined in step 411 that the condition is satisfied to takethe next sample of the target, then control passes to step 421 to usethe laser source to illuminate the target with a laser chirp for thechirp duration. In some embodiments, step 421 or a portion thereof isperformed by a hardware controller for the laser source; and, in someembodiments, step 421 or a portion thereof is performed by a generalpurpose processor programmed to control the laser source.

In step 423, a complex waveform is received at the one or more detectorsin the detector array at the fast frame rate. The complex waveformindicates both the amplitude and phase of the difference between thereturned signal from the target and a reference signal based on thelaser chirp, e.g., in heterodyne interference. In some embodiments, step423 also determines a range R to the target by determining the traveltime from the time of transmission of the first laser chirp to the timeof receipt of the first received laser chirp. In some embodiments, thisrange is used to set the delay time in the reference path 220 for thereference laser chirp for subsequent measurements of the complex fieldg₀(x, y, t_(f)). Step 423 is performed by the detector array, such as afast FPA. In some embodiments, a portion of step 423 is performed by ahardwired or programmable data acquisition system.

In step 425, the 3D field segment, i.e., g₀(x, y, t_(f)), is stored forthe current chirp in the current sampling event. In some embodiments,the data is natively 8-bit. Loading everything into memory and doing athree dimensional fast Fourier Transform (3D-FFT), which is a well-knowndigital procedure, would produce an array of 64 bit values (an 8-foldincrease in data load!) Therefore it is advantageous to do the 3D FFT ononly one or two dimensions at a time while only loading portions of thedata cube relevant for a particular step. Thus it is advantageous tostore the 3D cube elements in order of one dimension at a at time. Insome embodiments, step 425 or a portion thereof is performed by ahardware acquisition system; and, in some embodiments, step 425 or aportion thereof is performed by a general purpose processor programmedto perform data acquisition.

In step 427, it is determined whether there is another sample to betaken. If so, control passes back to step 411 and following steps,described above. If not, control passes to step 431. In someembodiments, step 427 or a portion thereof is performed by a hardwarecontroller for the laser source; and, in some embodiments, step 427 or aportion thereof is performed by a general purpose processor programmedto control the laser source. Thus step 427 includes repeating steps 421and 423 for multiple sampling conditions. In some embodiments, the arrayof one or more optical detectors is a subset of pixels in a digitalcamera. This offers the advantage of using commercial off the shelfcomponents that are much cheaper than an array of optical detectors ofother types. Such pixels are adequate in the various example embodimentsdescribed below, because the capture electronics can be concentrated onfewer pixels, thus allowing a frame rate for the subset of pixels to begreater than a frame rate for all the pixels in the digital camera. Inthese embodiments, repeating steps 421 and 423 for the multiple samplingconditions includes repeating steps 421 and 423 for multiple differentsubsets of the pixels in the digital camera.

In step 431, multiple 3D field segments are synthesized using aprocessor. In some embodiments, the field segments have fewer than threedimensions, e.g., when a single detector or a 1D array of detectors isused. Any conventional registration and synthesizing methods may beused. As indicated above, in some embodiments, the segments beingaperture synthesized are multiple subsets of pixels (regions ofinterest) of a digital camera. The no-physical motion aspect of theaperture synthesis is new. In prior attempts to expand HAL to threedimensions, as far as is known to the authors, nobody really thought touse a camera with subsets of pixels (regions of interest) as thedetector array combined with an aperture synthesis tool.

In some embodiments, novel methods are used for registering orsynthesizing the 3D segments. In some embodiments, spatial registrationis not needed because a fixed array is used with a stationary source,e.g., to watch the effects on an object of changes in values of one ormore environmental parameters like temperature, pressure, chemicalexposures; so spatial registration is assured by the physicalconfiguration. In other embodiments, registration is desirable andinvolves a similarity measure, such as cross-correlation, of specklefields to correctly register the segments. In some embodiments phasingof segments is done for both transverse dimension by performing aranging Fast Fourier Transform (FFT, a well known digital technique) ofa particular down-range segment (e.g., FFT the range dimension of the 3Darray for the segment). The phase at a chosen range bin containing abright point (or retro-reflector) in the scene is then extracted foreach transverse coordinate and this phase is removed from the segment atall range bins for each transverse coordinate. This process is repeatedacross all segments so the particular range bin always has zero phase atall transverse coordinates. Thus the various segments are phased in twotransverse dimensions. This causes the finally resolved scene to becentered on the particular bright point feature in 3D coordinate space;but, this situation is easily corrected by an appropriate circular shiftof the transverse array coordinates.

In some embodiments, 3D images are compared under different conditionsand the synthesizing is done for different sets of the field segments,i.e., one set of field segments are synthesized to produce one 3Drendering of an object under one set of conditions and another differentset of field segments are synthesized to produce another 3D rendering ofthe object under a different set of condition. For example, as shown inmore detail below, a coffee mug is compared before and after pouring inhot water, in order to show thermal expansion in a portion of the cup.Another example is when a symmetric object is being manufactured,asymmetries can be discovered by synthesizing one set of field segmentsrepresenting one side of the object and a different set of fieldsegments for a different side. If the different 3D renderings thatresult from the different sets show differences, then deviations fromthe desired symmetry have been exposed.

In step 433, a 3D Fourier transform is performed on at least one set ofthe synthesized 3D field segments. By virtue of step 433, the techniqueoffers un-ambiguous resolution of returns in all 3 dimensions. Thisoffers unforeseen advantages over other attempts to expand both SAL andHAL to three dimensions. Prior attempts to expand both SAL and HAL hadissues with phase wrapping and had to make some simplifying assumptionsabout the number of surfaces they are able to resolve in the “thirddimension”—usually just 1 surface. There is never a claim of full,Fourier limited, resolution of whatever may happen to be in any of thethree dimensions. Thus the type of target is not limited, or priorknowledge is not required, for the new techniques presented here. Themethod using the 3D fourier Transform can resolve targets that thoseprior techniques cannot resolve. People struggled with trying to get 3Ddata out of HAL for a while (more of a research area than 3D w/ SAL).

As mentioned above, it is advantageous to do a 3D FFT on only one or twodimensions at a time while only loading portions of the data cuberelevant for a particular step. Thus the data were stored efficientlyfor this purpose in step 425. In step 433, as the FFT of the firstdimension goes from real valued input data to complex valued data, thisproduces a 2-sided result. It is again advantageous to delete theredundant half of this data to minimize data overhead. For proper phaseextraction, it is still further advantageous to interpolate the peakusing a zero-padded FFT approach. Therefore, the range dimension wassubjected to a FFT last, one pair of transverse coordinates at a time.Commercial software can be used to implement the 3D-FFT. For example, inan illustrated embodiment, the FFT function fft(Array,[ ],n) availablein MATLAB™, from MATHWORKS™ of Natick Mass., was used, where n specifiesthe dimension and Array is a data array with at least a portion of the3D cube undergoing the transform. As pointed out above, the transformwas performed repeatedly, one dimension at a time, with care to savememory resources. Thus, in some embodiments, this step includesperforming a 3D Fourier transform of each synthesized by performing aone dimensional Fourier Transform of each dimension separately andcombining results from all dimensions.

In step 435, a digital representation, also called a digital model, ofthe target is generated based on the 3D Fourier transform. For example,as described above, a point cloud is extracted from the transformedspeckle field using one dimensional (1D) range analysis (per transverscoordinate) with a peak-finding fitting algorithm. The peak fits weredone with a 3-point Gaussian fit method, which is efficient relative toa nonlinear fit routine, because so many peaks had to be found (up toabout one million peaks per image). This made the extraction of gooddown-range precision much faster. The above processing was done in thenative coordinates provided by the array indexing. The array coordinatesare translated into target spatial coordinates depending on thedimension. Down-range coordinates were determined by multiplying byc/2B. Cross-range coordinates were determined by multiplying byδ_(CR)=λR/(2d_(SA)+d_(AP)). The size of the aperture was calculated bymultiplying the number of samples (in either cross range dimension) bythe pixel pitch of the detector array.

In step 437, a display device is operated based on the digitalrepresentation of the target. In various embodiments, the digital modelis a point cloud and the display device is a screen that displays thepoint cloud, or a system configured to render a surface from a pointcloud. In various embodiments, the display device is either a system,such as a surveillance system, configured to identify an object based onthe digital model, or a system, such as a robot, configured to operateon the target based on the digital model, or both.

High resolution 3D LADAR, and in particular 3D-HAL, relies upon aparallel readout architecture, it has advantages similar todirect-detect flash LADAR systems. However, because the approach iscoherent, it has many additional capabilities including: 1) enhancedcross-range resolution through aperture synthesis approaches; 2) accessto ultra-high downrange resolution through stretched processing ofchirped waveforms; 3) the ability to utilize digital phase correctionand image sharpness metrics for correcting optical or atmosphericaberrations; and 4) advanced Doppler or interferometric processing tomeasure the coherent evolution of the 3D point cloud. The 3D-HALapproach also uses heterodyne detection, which is extremely sensitivewhen shot-noise-limited, and which enables coherent integration toincrease carrier-to-noise ratios. This makes 3D-HAL ideally suited foruse in photon-starved environments typical of flash LADAR.

The use of coherent downrange waveforms provides unprecedentedextraction of 3D content. These 3D images are fully resolved and willperform better at 3D imaging targets with significant structure ordiscontinuities when compared to multi-wavelength HAL or multi-baselineSAL approaches.

2. EXAMPLE EMBODIMENTS 2.1 Simulated Embodiments

An advanced 3D-HAL model was developed to simulate a variety of systemarchitectures. Platform motion and/or target rotation has beenincorporated for aperture synthesis.

To demonstrate resolution enhancement, in one embodiment, a 3D-HALsystem was simulated to measure an AF bar target from a standoffdistance of 2 kilometers (km, 1 km=10³ meters). The bars target includeda series of horizontal and vertical bars of brushed aluminum on a planesloped at about 45 degrees. The bars range in diameter from about 0.05to 0.5 meters and in length from about 0.4 to about 4 meters. The targetalso includes the same material shaped into the numerals 0 and 1arranged both horizontally and vertically, and also shaped into lettersand numerals spelling out “USAF 1951 1X . . . ” The numerals and lettersare of similar sizes.

The fast FPA was assumed to be 200×280 pixels wide with a pixel spacingof 1.25 mm. Thirty-five total aperture locations were simulated with anaperture gain of 5 times in the vertical dimension and 7 times in thehorizontal dimension in the plane perpendicular to the range to thetarget. This corresponds to using a sliding window of pixels in acommercial 2 Megabyte digital camera. FIG. 5A is a block diagram thatillustrates an example 2D array of optical detectors forming a 2Dsynthetic aperture, according to an embodiment. One pupil-plane fieldsegment includes 200 by 280 pixels and the 5×7 grid of segments visitedat different times, each time subjected to one laser chirp, produces thesynthetic array of 1000 vertical pixels by 1960 horizontal pixels. Noregistration was needed because the data were actually collected on the1000×1960 pixel array. In some simulated embodiments, simulatedpositional uncertainty is added and scene or speckle or overlapregistration is used. Each tile was collected at a different time,serially, with B≈90 GHz, D≈200 ms, and a 5 Hz chirp repetition rate.

FIG. 5B is a plot that illustrates an example 3D image using a singleinstance of the 2D array of FIG. 5A, according to an embodiment. Thusthis shows a 3D rendering for a single pupil-plane segment. Individualbars are not resolved and no letters or numerals are recognizable. FIG.5C is a plot that illustrates an example 3D image using multipleinstances of the 2D array of FIG. 5A in a synthetic aperture, accordingto an embodiment. Thus this shows the full aperture synthesis anddemonstrates the anticipated resolution enhancement in both transversedimensions. Individual bars are resolved and both letters and numeralsare recognizable with good depth resolution commensurate with theenhanced horizontal resolution, showing the efficacy of the 3D LADARapproach.

2.2 Experimental Embodiments

D-HAL demonstrations were conducted using a lens-less digitalholographic setup. A stabilized FMCW linear-chirp laser centered at acarrier wavelength of 1064 nanometers (nm, 1 nm=10⁻⁹ meters) was used toflood illuminate various targets 2.5 meters away, each of which wasmounted on a rotary stage. The laser output power was approximately 15milliWatts (mW, 1 mW=10⁻³ watts). The linear chirp had a bandwidth of102 GHz and was repeated at a 5 Hz rate (i.e., the chirp had a durationD of 0.2 seconds=200 milliseconds, ms, 1 ms=10⁻³ seconds). This laserwas an SLM-L Source, available from BRIDGER PHOTONICS™, Bozeman, Mont.Such chirps also have been demonstrated in various publications, such asKrause et al., 2012, Satyan et al., 2009, and Kewitsch, et al., 2006.

A mirror near the target was used as reference path 220 to create alocal oscillator beam as the reference beam 207 b. A Basler acA2000-340km CMOS array used in many digital cameras served as detector array 230and recorded the interference between the reference beam and the targetreturns synchronized with the chirp output. While the CMOS arrayincluded 1000×1960 pixels, only a 200×280 pixel region of interest wassampled during any 200 ms sampling period, i.e., duration D. A BitflowKarbon frame grabber card was used to capture the 200×280 pixelregion-of-interest (ROI) at a frame rate of 1.8 kHz—much faster than aframe rate needed to capture the almost 2 million pixels of the entirearray. The 2D synthesis accounts for the different time each different200×280 pixel subset was sampled. Thus, each resulting field segment had360 frames in fast time and was 200 ms in duration. The target wasstationary during the entire capture of the 1000×1960 pixel image set(taken in 200×280 pixel segments serially). Then the target was rotatedto get a new speckle realization and the full capture process was againrepeated.

The ROI was shifted to each of the 35 non-overlapping positions on the1000×1960 CMOS array, similar to the tiles depicted in FIG. 5A, in orderto simulate synthetic aperture. This is effectively a bi-static modewith a moving receive aperture and stationary transmitter both withfixed pointing. The bi-static mode halves the synthetic aperture anddoubles the resolution. However, the stationary fast FPA results inideal spatial placement of the field segments and simplifies theprocessing so spatial registration is not needed. The resultant voxel is1.5 mm×968 μm×494μm.

The tiles produce segments that are subjected to piston phasing, asdescribed here, achieved with a ball lens retro-reflector. A N-BK7 index2.0 ball lens (available from EDMOND OPTICS INC.™ of Barrington, N.J.)was placed in the scene to provide strong reflection. The phase on thereturn from this reflection was then analyzed to provide segment tosegment phase offset compensation via digital processing. Given thepulse repetition frequency of 5 Hz (chirp duration of 200 ms), thecollection time of a data cube is roughly 7 seconds. Several data cubescan allow for coherent integration of the time domain signal forincreased SNR. The segments are coherently added (3D Array sum ofcomplex values in amplitude and phase) after piston phase removal(described above). The coherent summation increases signal to noiseratio (SNR). Equation for N samples of a given segment would beSegment_(coherent sum)=Segment_(sample1)+Segment_(sample2)+ . . .+Segment_(sampleN) Where the “+” is an elementwise addition of complexnumbers.

A rotary stage rotated the target by the angle subtended by the FPA fromthe target to achieve an independent speckle realization. When thetarget is rotated to provide a new speckle realization, the portions ofthe target which are not on the rotation axis (most of it) translate inrange and cross range slightly—an effect called surface migration. Dueto the small amount of rotation however, this surface migration was muchless than a resolution bin and was therefore negligible in theexperimental embodiment.

The data shown below are derived from four speckle realizations of thefull data cube. Though target rotation has been successfully used foraperture synthesis, the additional information for speckle averaging wasused to improve image contrast. For example, with target rotation, theassociated speckle field translation could have been used for aperturesynthesis. This would have involved actual algorithmic fieldregistration, which would have led to higher cross-range resolution inthe dimension of the aperture synthesis. However, as the imagingtechnique is coherent, there is speckle induced range noise andintensity noise. Thus use of the target rotation to achieve independentestimates of the speckle fields and subsequent speckle averaging wasdeemed to improve the overall image quality more favorably, withsmoother, more consistent intensity, than the image quality attainedwith improved cross-range resolution via aperture synthesis.

2.2.1 Experimental Rendering of Sheet Surface

To demonstrate down-range resolution enhancement, a satin finishedaluminum plate with precision machined features was imaged. FIG. 6A is aphotograph that illustrates an example surface used as a target,according to an embodiment. The machined features included vertical andhorizontal grooves at three different widths (300, 600, and 900μm) andtwo different depths (100 and 200μm). A corner of the aluminum plate wasremoved at a depth of 500 μm to assess the “step” resolution of thesystem. Holes of varying diameters (1, 4, 6 and 10 mm) were also drilledthrough the plate. The 3D image was formed using a single field segmentas well as the entire coherently stitched array to demonstrate theresolution enhancement capability. FIG. 6B is a plot that illustrates anexample 3D image of the surface of FIG. 6A using a single instance of a2D array, according to an embodiment. The depth to the point cloud isindicated by grey-shading. Individual features are not recognizable.FIG. 6C is a plot that illustrates an example 3D image of the surface ofFIG. 6A using multiple instances of the 2D array in a syntheticaperture, according to an embodiment. The point cloud derived fromprocessing of the fully synthesized aperture of size 1000×1960 withdepth out of the plane given by greyscale. The visualization shows thecross range resolution enhancement of the stitched field versus thesingle segment. The features and quantitative estimates of depth aerecognizable.

2.2.2 Experimental Rendering of Razor

FIG. 7A is a photograph that illustrates an example object used as atarget, according to an embodiment. This shaving razor target was chosento provide different material and shape characteristics using a familiarobject. FIG. 7B is a plot that illustrates an example 3D image of theobject of FIG. 7A using a single instance of a 2D array, according to anembodiment. This shows the single field segment. FIG. 7C is a plot thatillustrates an example 3D image of the object of FIG. 7A using multipleinstances of the 2D array in a synthetic aperture, according to anembodiment. This shows the fully synthesized aperture 3D-HAL imageryrendered with log scale intensity shading. The synthetic point cloudsprovide more detail, allowing for example the grip on the razor to beresolved.

2.2.3 Experimental Rendering of Cup and Straw

FIG. 8A is a photograph that illustrates another example object used asa target, according to an embodiment. This 12 once paper cup ad strawtarget was chosen to provide different material and shapecharacteristics using another familiar object. FIG. 8B is a plot thatillustrates an example 3D image of the object of FIG. 8A using a singleinstance of a 2D array, according to an embodiment. FIG. 8C is a plotthat illustrates an example 3D image of the object of FIG. 8A usingmultiple instances of the 2D array in a synthetic aperture, according toan embodiment.

2.2.4 Experimental Detection of Thermal Stress

To fully demonstrate the coherent nature of point clouds rendered withthe 3D-HAL approach, the phase information of the various points wastracked over sequential captures. Because this phase is proportional tothe wavelength of the optical carrier, very small scale changes can beobserved and mapped to the 3D rendering. FIG. 9A is a photograph thatillustrates another example object used as a target, according to anembodiment. To explore this capability, the thermal expansion of acoffee cup in the seconds after hot water was introduced into thecontainer was imaged. FIG. 9A show the coffee mug and the boiling waterlevel.

The ROI was fixed at 400×560 pixels, with 2×2 binning, resulting in anarrower field-of-view, which allowed a 5 Hz frame rate. The phase valueof each voxel over the sequential capture was then referenced to thephase of the identical voxel in the first capture. This phase wasunwrapped and rendered onto a surface reconstruction of the point cloudto visualize thermal expansion of the mug on the micron scale.

The results are shown in FIG. 9B. FIG. 9B is an image that illustratesan example distribution of differences in the object of FIG. 9A atdifferent times due to different environmental conditions, according toan embodiment. A grayscale of the unwrapped phase after 60 seconds isrenders rendered onto the point cloud representing the surface of themug. The observed phase evolution demonstrates that more thermalexpansion is occurring in the bottom half of the mug where the hot wateris present. In order to correctly utilize this phase to measuredisplacement, the surface normal is estimated and compared to the lineof sight of the 3D-HAL system as shown in FIG. 9C. FIG. 9C is a plotthat illustrates components of expansion toward and away from an arrayof optical detectors, according to an embodiment.

The observed phase change is proportional to the downward pointingarrows, which represent the dot product of the thermal expansion(exaggerated) indicated by the lettered arrows A and B along the line ofsight of the 3D-HAL system. By taking this projection into account, anestimate of the displacement can be made. FIG. 9D is an image thatillustrates an example distribution of differences in the object of FIG.9A at different times due to different environmental conditionscorrected for assumed direction of expansion, according to anembodiment. As can be seen thermal expansion occurs along he entire bandbelow the surface of the boiling water. The fully coherent nature of thepoint clouds is demonstrated by such tracking of the phase of athermally expanding coffee cup.

3. COMPUTATIONAL HARDWARE OVERVIEW

FIG. 10 is a block diagram that illustrates a computer system 1000 uponwhich an embodiment of the invention may be implemented. Computer system1000 includes a communication mechanism such as a bus 1010 for passinginformation between other internal and external components of thecomputer system 1000. Information is represented as physical signals ofa measurable phenomenon, typically electric voltages, but including, inother embodiments, such phenomena as magnetic, electromagnetic,pressure, chemical, molecular atomic and quantum interactions. Forexample, north and south magnetic fields, or a zero and non-zeroelectric voltage, represent two states (0, 1) of a binary digit (bit).).Other phenomena can represent digits of a higher base. A superpositionof multiple simultaneous quantum states before measurement represents aquantum bit (qubit). A sequence of one or more digits constitutesdigital data that is used to represent a number or code for a character.In some embodiments, information called analog data is represented by anear continuum of measurable values within a particular range. Computersystem 1000, or a portion thereof, constitutes a means for performingone or more steps of one or more methods described herein.

A sequence of binary digits constitutes digital data that is used torepresent a number or code for a character. A bus 1010 includes manyparallel conductors of information so that information is transferredquickly among devices coupled to the bus 1010. One or more processors1002 for processing information are coupled with the bus 1010. Aprocessor 1002 performs a set of operations on information. The set ofoperations include bringing information in from the bus 1010 and placinginformation on the bus 1010. The set of operations also typicallyinclude comparing two or more units of information, shifting positionsof units of information, and combining two or more units of information,such as by addition or multiplication. A sequence of operations to beexecuted by the processor 1002 constitutes computer instructions.

Computer system 1000 also includes a memory 1004 coupled to bus 1010.The memory 1004, such as a random access memory (RAM) or other dynamicstorage device, stores information including computer instructions.Dynamic memory allows information stored therein to be changed by thecomputer system 1000. RAM allows a unit of information stored at alocation called a memory address to be stored and retrievedindependently of information at neighboring addresses. The memory 1004is also used by the processor 1002 to store temporary values duringexecution of computer instructions. The computer system 1000 alsoincludes a read only memory (ROM) 1006 or other static storage devicecoupled to the bus 1010 for storing static information, includinginstructions, that is not changed by the computer system 1000. Alsocoupled to bus 1010 is a non-volatile (persistent) storage device 1008,such as a magnetic disk or optical disk, for storing information,including instructions, that persists even when the computer system 1000is turned off or otherwise loses power.

Information, including instructions, is provided to the bus 1010 for useby the processor from an external input device 1012, such as a keyboardcontaining alphanumeric keys operated by a human user, or a sensor. Asensor detects conditions in its vicinity and transforms thosedetections into signals compatible with the signals used to representinformation in computer system 1000. Other external devices coupled tobus 1010, used primarily for interacting with humans, include a displaydevice 1014, such as a cathode ray tube (CRT) or a liquid crystaldisplay (LCD), for presenting images, and a pointing device 1016, suchas a mouse or a trackball or cursor direction keys, for controlling aposition of a small cursor image presented on the display 1014 andissuing commands associated with graphical elements presented on thedisplay 1014.

In the illustrated embodiment, special purpose hardware, such as anapplication specific integrated circuit (IC) 1020, is coupled to bus1010. The special purpose hardware is configured to perform operationsnot performed by processor 1002 quickly enough for special purposes.Examples of application specific ICs include graphics accelerator cardsfor generating images for display 1014, cryptographic boards forencrypting and decrypting messages sent over a network, speechrecognition, and interfaces to special external devices, such as roboticarms and medical scanning equipment that repeatedly perform some complexsequence of operations that are more efficiently implemented inhardware.

Computer system 1000 also includes one or more instances of acommunications interface 1070 coupled to bus 1010. Communicationinterface 1070 provides a two-way communication coupling to a variety ofexternal devices that operate with their own processors, such asprinters, scanners and external disks. In general the coupling is with anetwork link 1078 that is connected to a local network 1080 to which avariety of external devices with their own processors are connected. Forexample, communication interface 1070 may be a parallel port or a serialport or a universal serial bus (USB) port on a personal computer. Insome embodiments, communications interface 1070 is an integratedservices digital network (ISDN) card or a digital subscriber line (DSL)card or a telephone modem that provides an information communicationconnection to a corresponding type of telephone line. In someembodiments, a communication interface 1070 is a cable modem thatconverts signals on bus 1010 into signals for a communication connectionover a coaxial cable or into optical signals for a communicationconnection over a fiber optic cable. As another example, communicationsinterface 1070 may be a local area network (LAN) card to provide a datacommunication connection to a compatible LAN, such as Ethernet. Wirelesslinks may also be implemented. Carrier waves, such as acoustic waves andelectromagnetic waves, including radio, optical and infrared wavestravel through space without wires or cables. Signals include man-madevariations in amplitude, frequency, phase, polarization or otherphysical properties of carrier waves. For wireless links, thecommunications interface 1070 sends and receives electrical, acoustic orelectromagnetic signals, including infrared and optical signals, whichcarry information streams, such as digital data.

The term computer-readable medium is used herein to refer to any mediumthat participates in providing information to processor 1002, includinginstructions for execution. Such a medium may take many forms,including, but not limited to, non-volatile media, volatile media andtransmission media. Non-volatile media include, for example, optical ormagnetic disks, such as storage device 1008. Volatile media include, forexample, dynamic memory 1004. Transmission media include, for example,coaxial cables, copper wire, fiber optic cables, and waves that travelthrough space without wires or cables, such as acoustic waves andelectromagnetic waves, including radio, optical and infrared waves. Theterm computer-readable storage medium is used herein to refer to anymedium that participates in providing information to processor 1002,except for transmission media.

Common forms of computer-readable media include, for example, a floppydisk, a flexible disk, a hard disk, a magnetic tape, or any othermagnetic medium, a compact disk ROM (CD-ROM), a digital video disk (DVD)or any other optical medium, punch cards, paper tape, or any otherphysical medium with patterns of holes, a RAM, a programmable ROM(PROM), an erasable PROM (EPROM), a FLASH-EPROM, or any other memorychip or cartridge, a carrier wave, or any other medium from which acomputer can read. The term non-transitory computer-readable storagemedium is used herein to refer to any medium that participates inproviding information to processor 1002, except for carrier waves andother signals.

Logic encoded in one or more tangible media includes one or both ofprocessor instructions on a computer-readable storage media and specialpurpose hardware, such as ASIC 1020.

Network link 1078 typically provides information communication throughone or more networks to other devices that use or process theinformation. For example, network link 1078 may provide a connectionthrough local network 1080 to a host computer 1082 or to equipment 1084operated by an Internet Service Provider (ISP). ISP equipment 1084 inturn provides data communication services through the public, world-widepacket-switching communication network of networks now commonly referredto as the Internet 1090. A computer called a server 1092 connected tothe Internet provides a service in response to information received overthe Internet. For example, server 1092 provides information representingvideo data for presentation at display 1014.

The invention is related to the use of computer system 1000 forimplementing the techniques described herein. According to oneembodiment of the invention, those techniques are performed by computersystem 1000 in response to processor 1002 executing one or moresequences of one or more instructions contained in memory 1004. Suchinstructions, also called software and program code, may be read intomemory 1004 from another computer-readable medium such as storage device1008. Execution of the sequences of instructions contained in memory1004 causes processor 1002 to perform the method steps described herein.In alternative embodiments, hardware, such as application specificintegrated circuit 1020, may be used in place of or in combination withsoftware to implement the invention. Thus, embodiments of the inventionare not limited to any specific combination of hardware and software.

The signals transmitted over network link 1078 and other networksthrough communications interface 1070, carry information to and fromcomputer system 1000. Computer system 1000 can send and receiveinformation, including program code, through the networks 1080, 1090among others, through network link 1078 and communications interface1070. In an example using the Internet 1090, a server 1092 transmitsprogram code for a particular application, requested by a message sentfrom computer 1000, through Internet 1090, ISP equipment 1084, localnetwork 1080 and communications interface 1070. The received code may beexecuted by processor 1002 as it is received, or may be stored instorage device 1008 or other non-volatile storage for later execution,or both. In this manner, computer system 1000 may obtain applicationprogram code in the form of a signal on a carrier wave.

Various forms of computer readable media may be involved in carrying oneor more sequence of instructions or data or both to processor 1002 forexecution. For example, instructions and data may initially be carriedon a magnetic disk of a remote computer such as host 1082. The remotecomputer loads the instructions and data into its dynamic memory andsends the instructions and data over a telephone line using a modem. Amodem local to the computer system 1000 receives the instructions anddata on a telephone line and uses an infra-red transmitter to convertthe instructions and data to a signal on an infra-red a carrier waveserving as the network link 1078. An infrared detector serving ascommunications interface 1070 receives the instructions and data carriedin the infrared signal and places information representing theinstructions and data onto bus 1010. Bus 1010 carries the information tomemory 1004 from which processor 1002 retrieves and executes theinstructions using some of the data sent with the instructions. Theinstructions and data received in memory 1004 may optionally be storedon storage device 1008, either before or after execution by theprocessor 1002.

FIG. 11 illustrates a chip set 1100 upon which an embodiment of theinvention may be implemented. Chip set 1100 is programmed to perform oneor more steps of a method described herein and includes, for instance,the processor and memory components described with respect to FIG. 10incorporated in one or more physical packages (e.g., chips). By way ofexample, a physical package includes an arrangement of one or morematerials, components, and/or wires on a structural assembly (e.g., abaseboard) to provide one or more characteristics such as physicalstrength, conservation of size, and/or limitation of electricalinteraction. It is contemplated that in certain embodiments the chip setcan be implemented in a single chip. Chip set 1100, or a portionthereof, constitutes a means for performing one or more steps of amethod described herein.

In one embodiment, the chip set 1100 includes a communication mechanismsuch as a bus 1101 for passing information among the components of thechip set 1100. A processor 1103 has connectivity to the bus 1101 toexecute instructions and process information stored in, for example, amemory 1105. The processor 1103 may include one or more processing coreswith each core configured to perform independently. A multi-coreprocessor enables multiprocessing within a single physical package.Examples of a multi-core processor include two, four, eight, or greaternumbers of processing cores. Alternatively or in addition, the processor1103 may include one or more microprocessors configured in tandem viathe bus 1101 to enable independent execution of instructions,pipelining, and multithreading. The processor 1103 may also beaccompanied with one or more specialized components to perform certainprocessing functions and tasks such as one or more digital signalprocessors (DSP) 1107, or one or more application-specific integratedcircuits (ASIC) 1109. A DSP 1107 typically is configured to processreal-world signals (e.g., sound) in real time independently of theprocessor 1103. Similarly, an ASIC 1109 can be configured to performedspecialized functions not easily performed by a general purposedprocessor. Other specialized components to aid in performing theinventive functions described herein include one or more fieldprogrammable gate arrays (FPGA) (not shown), one or more controllers(not shown), or one or more other special-purpose computer chips.

The processor 1103 and accompanying components have connectivity to thememory 1105 via the bus 1101. The memory 1105 includes both dynamicmemory (e.g., RAM, magnetic disk, writable optical disk, etc.) andstatic memory (e.g., ROM, CD-ROM, etc.) for storing executableinstructions that when executed perform one or more steps of a methoddescribed herein. The memory 1105 also stores the data associated withor generated by the execution of one or more steps of the methodsdescribed herein.

4. ALTERATIONS, EXTENSIONS AND MODIFICATIONS

In the foregoing specification, the invention has been described withreference to specific embodiments thereof. It will, however, be evidentthat various modifications and changes may be made thereto withoutdeparting from the broader spirit and scope of the invention. Thespecification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. Throughout thisspecification and the claims, unless the context requires otherwise, theword “comprise” and its variations, such as “comprises” and“comprising,” will be understood to imply the inclusion of a stateditem, element or step or group of items, elements or steps but not theexclusion of any other item, element or step or group of items, elementsor steps. Furthermore, the indefinite article “a” or “an” is meant toindicate one or more of the item, element or step modified by thearticle.

5. REFERENCES

The following references are hereby incorporated by reference as iffully set forth herein except for terminology that is inconsistent withthe terminology used herein.

-   1. Aull, B. F., Loomis, A. H., Young, D. J., Heinrichs, R. M.,    Felton, B. J., Daniels, P. J. and Landers, D. J., “Geigermode    avalanche photodiodes for three-dimensional imaging,” Lincoln    Lab. J. 13, 335-350 (2002).-   2. Beck, S. M., Buck, J. R., Buell, W. F., Dickinson, R. P.,    Kozlowski, D. A., Marechal, N. J., and Wright, T. J.,    “Synthetic-aperture imaging laser radar: laboratory demonstration    and signal processing,” Appl. Opt. 44, 7621-7629 (2005).-   3. Berkovic, G. and Shafir E., “Optical methods for distance and    displacement measurements,” Adv. Opt. Photon. 4, 441-471 (2012).-   4. Crouch, S. and Barber, Z. W., “Laboratory demonstrations of    interferometric and spotlight synthetic aperture ladar techniques,”    Opt. Express 20, 24237-24246 (2012).-   5. Dapore, B. R., Rabb, D. J., and Haus, J. W., “Phase noise    analysis of two wavelength coherent imaging system,” Opt. Express    21, 30642-30652 (2013).-   6. Duncan, B. D. and Dierking, M. P., “Holographic aperture ladar:    erratum,” Appl. Opt. 52, 706-708 (2013).-   7. Kewitsch, A., Rakuljic, G., Yariv, A., Semiconductor lasers in    optical phase-locked loops, WIPO publication WO 2007124063, US    Patent Office publication US20060239312-   8. Krause, Brian W., Tiemann, Bruce G., and Gatt, Philip, “Motion    compensated frequency modulated continuous wave 3D coherent imaging    ladar with scannerless architecture,” Appl. Opt. v51, pp 8745-8761    (2012)-   9. Marron, J. C. and Schroeder, K. S., “Three-dimensional lensless    imaging using laser frequency diversity,” Appl. Opt. 31, 255-262    (1992).-   10. Rabb, D. J., Jameson, D. F., Stafford, J. W., and Stokes, A. J.,    “Multi-transmitter aperture synthesis,” Opt. Express 18, 24937-24945    (2010).-   11. Roos, P. A., Reibel, R. R., Berg, T., Kaylor, B., Barber, Z. W.,    and Babbitt, W. R., “Ultrabroadband optical chirp linearization for    precision metrology applications,” Opt. Lett. 34, 3692-3694 (2009).-   12. Satyan, N., Vasilyev, A., Rakuljic, G., Leyva, V., and Yariv, A,    “Precise control of broadband frequency chirps using optoelectronic    feedback,” Opt. Express 17, 15991-15999 (2009)-   13. Stafford, J., Duncan, B., and Rabb, D. J., “Range-compressed    Holographic Aperture Ladar,” in Imaging and Applied Optics 2015, OSA    Technical Digest (online) (Optical Society of America, 2015), paper    LM1F.3.-   14. Tippie, A. E., Kumar, A., and Fienup, J. R., “High-resolution    synthetic-aperture digital holography with digital phase and pupil    correction,” Optics Express 19 (13), 12027-12038 (2011).

What is claimed is:
 1. A method comprising: a) splitting a lasertemporally-modulated waveform of bandwidth B and duration D from a lasersource into a reference beam and a target beam; b) directing the targetbeam onto a target; c) collecting first data that indicates amplitudeand phase of light relative to the reference beam received at each of aplurality of different times during a duration D at each opticaldetector of an array of one or more optical detectors in a planeperpendicular to the target beam; d) repeating steps a, b and c for aplurality of sampling conditions; e) synthesizing the first data for theplurality of sampling conditions to form one or more synthesized sets;f) performing a 3D Fourier transform of each synthesized set to form adigital model of the target for each synthesized set with a down-rangeresolution based on the bandwidth B; and g) operating a display devicebased at least in part on at least a portion of the digital model of thetarget for at least one synthesized set.
 2. A method as recited in claim1, further comprising determining an average range to the target basedon a travel time of a laser pulse reflected from the target andproviding a reference path length for the reference beam based on theaverage range to the target.
 3. A method as recited in claim 1, whereinthe digital model is a point cloud and the display device is a systemconfigured to render a surface from a point cloud.
 4. A method asrecited in claim 1, wherein the display device is a system configured toidentify an object based on the digital model.
 5. A method as recited inclaim 1, wherein the display device is a system configured to operate onthe target based on the digital model.
 6. A method as recited in claim1, wherein the plurality of sampling conditions are a plurality ofdifferent angles from the target to the array of one or more opticaldetectors.
 7. A method as recited in claim 1, wherein the plurality ofsampling conditions is a plurality of different times while the targetis subjected to a change in environment.
 8. The method as recited inclaim 1, wherein the one or more synthesized sets includes at least twosynthesized sets and the step of operating the display device furthercomprises operating the display device to present second data thatindicates a difference between at least two different digital modelsformed from the at least two synthesized sets.
 9. The method as recitedin claim 8, wherein the at least two synthesized sets represent shape ofan object for at least two different sampling conditions.
 10. A methodas recited in claim 9, wherein the plurality of sampling conditions area plurality of different times while the target is subjected to a changein environment.
 11. The method as recited in claim 10 wherein the changein environment is a change in thermal conditions and the differencebetween the at least two different digital models indicates thermalexpansion in response to the change in thermal conditions.
 12. Themethod as recited in claim 11 wherein the thermal expansion is in arange from about 1 micron to about 100 microns.
 13. The method asrecited in claim 1, wherein synthesizing the first data furthercomprises, for each synthesized set, selecting a plurality of subsets ofthe first data, synthesizing each subset separately to produce asynthesized subset and incoherently combining the plurality ofsynthesized subsets.
 14. The method as recited in claim 1, whereinperforming a 3D Fourier transform of each synthesized set furthercomprises performing a one dimensional Fourier Transform of eachdimension separately and combining results from all dimensions.
 15. Themethod as recited in claim 1, wherein: the array of one or more opticaldetectors is a subset of pixels in a digital camera to allow a framerate for the subset of pixels to be greater than a frame rate for allthe pixels in the digital camera; and said step d of repeating steps a,b, and c for the plurality of sampling conditions further comprisesrepeating steps a, b, and c for a plurality of different subsets of thepixels in the digital camera.
 16. A non-transitory computer-readablemedium carrying one or more sequences of instructions, wherein executionof the one or more sequences of instructions by one or more processorscauses the one or more processors to perform the steps of: receiving,for a plurality of sampling conditions, first data that indicatesamplitude and phase of light, relative to a reference beam, receivedfrom a target, illuminated by a target beam from a lasertemporally-modulated waveform source for a duration D, at each of aplurality of different times during the duration D at each opticaldetector of an array of one or more optical detectors in a planeperpendicular to the target beam; synthesizing the first data for theplurality of sampling conditions to form one or more synthesized sets;performing a 3D Fourier transform of each synthesized set to form adigital model of the target for each synthesized set; and presentingoutput data based at least in part on at least a portion of the digitalmodel of the target for at least one synthesized set.
 17. An apparatuscomprising: at least one processor; and at least one memory includingone or more sequences of instructions, the at least one memory and theone or more sequences of instructions configured to, with the at leastone processor, cause the apparatus to perform at least the following,receiving, for a plurality of sampling conditions, first data thatindicates amplitude and phase of light, relative to a reference beam,received from a target, illuminated by a target beam from a lasertemporally-modulated waveform source for a duration D, at each of aplurality of different times during the duration D at each opticaldetector of an array of one or more optical detectors in a planeperpendicular to the target beam; synthesizing the first data for theplurality of sampling conditions to form one or more synthesized sets;performing a 3D Fourier transform of each synthesized set to form adigital model of the target for each synthesized set; and presentingoutput data based at least in part on at least a portion of the digitalmodel of the target for at least one synthesized set.
 18. A systemcomprising: a laser temporally-modulated waveform source configured toproduce a target beam and a reference beam each with a bandwidth B andduration D; an array of one or more optical detectors in a planeperpendicular to the target beam; a display device; at least oneprocessor; and at least one memory including one or more sequences ofinstructions, the at least one memory and the one or more sequences ofinstructions configured to, with the at least one processor, cause theat least one processor to perform at least the following, collecting,for a plurality of sampling conditions, first data that indicatesamplitude and phase of light relative to the reference beam received ateach of a plurality of different times during the duration D at eachoptical detector of the array of one or more optical detectors;synthesizing the first data for the plurality of sampling conditions toform one or more synthesized sets; performing a 3D Fourier transform ofeach synthesized set to form a digital model of the target for eachsynthesized set; and operating the display device based at least in parton at least a portion of the digital model of the target for at leastone synthesized set.